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Dear  Mr.  Chairman: 

This  report  responds  to  your  request  that  we  review  the  Strategic  Defense  Initiative 
Organization’s  analyses  of  the  effectiveness  of  Brilliant  Pebbles,  the  proposed  space-based 
weapon  for  the  Global  Protection  Against  Limited  Strikes  (GPALS)  Strategic  Defense  System.  The 
report  discusses  the  role  of  computer  simulations  in  assessing  the  effectiveness  of  the  Brilliant 
Pebbles  system. 

We  are  sending  copies  of  this  report  to  appropriate  congressional  committees,  the  Secretaries  of 
Defense  and  the  Air  Force,  ar  d  the  Directors,  Strategic  Defense  Initiative  Organization  and 
Office  of  Management  and  Budget.  We  will  also  make  copies  available  to  others. 

Please  contact  me  at  (202)  275-4268  if  you  or  your  staff  have  any  questions  concerning  this 
report.  Major  contributors  are  listed  in  appendix  II. 

Sincerely  yours, 


Director 
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Executive  Summary 


Purpose 


In  January  1991 ,  the  President  directed  that  the  Strategic  Defense  Initiative 
(sdi)  program  be  refocused  toward  providing  protection  against  limited 
ballistic  missile  strikes,  whether  deliberate,  accidental,  or  unauthorized. 
The  proposed  system  concept  is  known  as  Global  Protection  Against 
Limited  Strikes,  or  GPALS,  and  would  consist  of  both  surface-  and 
space-based  sensors  and  interceptors.  Because  this  was  a  significant 
change  in  the  program,  the  Strategic  Defense  Initiative  Organization  (SDIO) 
and  its  contractors  are  still  adapting  the  sdi  program  to  the  directive  and 
are  assessing  a  proposed  space-based  interceptor  system  known  as 
Brilliant  Pebbles. 

The  Chairman,  Senate  Armed  Services  Committee,  requested  that  GAO 
examine  SDIO’s  calculation  of  Brilliant  Pebbles’  contribution  to  this  new 
mission  and  identify  the  critical  assumptions  and  uncertainties  contained  in 
SDIO’s  analysis  of  Brilliant  Pebbles’  effectiveness. 


Background 


y- 


Brilliant  Pebbles  is  a  proposed  concept  that  is  currently  in  the  early  stages 
of  its  demonstration  and  validation  phase  of  development.  It  entails 
hundreds  of  individual  interceptors  in  orbit  around  the  earth  at  relatively 
evenly  spaced  intervals.  Each  interceptor  would  be  linked  by 
communications  to  the  others  and  to  ground  stations.  In  the  event  of  a 
ballistic  missile  attack,  each  could  be  given  a  high  degree  of  autonomy  to 
detect  and  intercept  missiles  that  enter  its  battle  space. 

A  set  of  deployed  Brilliant  Pebbles,  referred  to  as  a  constellation,  would  be 
made  up  of  several  staggered  rings  orbiting  at  about  400  kilometers  above 
the  earth,  with  several  Brilliant  Pebbles  in  each  ring.  (See  fig.  1.1.)  The 
constellation  could  be  deployed  either  to  provide  partial  or  complete  global 
coverage  for  detection  and  interception  of  ballistic  missiles  on  a 
continuous  basis.  Once  enabled  by  human  command,  the  Brilliant  Pebbles 
interceptors  could  select  their  targets  and  divert  from  their  orbits  into  the 
path  of  enemy  missiles.  The  interceptors  would  carry  no  explosives,  but 
the  force  of  their  high-speed  collision  is  expected  to  destroy  targets. 

SDia’s  estimates  of  the  overall  effectiveness  of  a  Brilliant  Pebbles 
space-based  interceptor  are  based  on  computer  simulations,  which  at  this 
early  stage  of  the  program’s  development  are  the  only  means  available  to 
estimate  performance  of  a  constellation  of  Brilliant  Pebbles  against  a 
ballistic  missile  attack. 
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Results  in  Brief 


During  this  early  stage  of  Brilliant  Pebbles’  development,  SDIO  has  made 
extensive  use  of  simulations  to  answer  the  question:  If  SDIO  can  design, 
manufacture  and  deploy  a  system  that  functions  as  planned,  will  it  provide 
the  protection  desired?  SDIO  believes  simulations  answer  that  question  in 
the  affirmative  and  uses  the  results  to  improve  the  design. 

Congress  should  be  aware,  however,  of  the  simulations’  limitations.  The 
simulations  are  still  immature  and  use  many  unproven  assumptions.  They 
do  not  demonstrate  that  Brilliant  Pebbles  can  be  built  and  will  work.  Only 
later  in  the  development  cycle,  after  Brilliant  Pebbles  has  been  fabricated 
and  tested,  can  the  assumptions  be  replaced  with  data  from  testing,  sdio 
has  an  extensive  test  program  planned  over  5  years  to  gather  data.  As 
testing  results  replace  assumptions,  SDIO  and  Congress  can  then  have 
increased  confidence  in  the  simulations’  projections  of  Brilliant  Pebbles’ 
effectiveness. 


Principal  Findings 


Projected  Effectiveness 
Based  on  Computer 
Simulations 


sdio’s  estimates  of  effectiveness  are  based  on  computer  simulations  of 
various  numbers  of  interceptors  deployed  against  certain  hypothetical 
ballistic  missile  attacks.  SDIO  has  identified  over  40  hypothetical  attack 
scenarios,  or  threats,  against  the  United  States  and  its  allies,  which  include 
short-,  intermediate-,  and  long-range  ballistic  missile  attacks  originating 
from  all  over  the  world  and  submarine  launched  attacks  against  the  United 
States.  SDIO  has  investigated  many  potential  deployment  schemes  to 
identify  a  constellation  that  provides  the  optimum  global  protection  against 
all  threats.  As  of  December  1991,  SDIO  had  not  evaluated  through 
simulations  the  performance  of  Brilliant  Pebbles  against  all  identified 
threats.  SDIO  officials  told  us  subsequently  that  they  completed  this 
evaluation. 


Deployment  Assumptions  SDIO’S  computer  simulations  contain  deployment  decision  assumptions 
Impact  Effectiveness  such  as  the  number  of  Brilliant  Pebbles  in  the  constellation  and  the  angle  at 

which  the  rings  cross  the  equator.  Since  the  refocus  to  GPALS,  SDIO  has 
performed  simulations  in  which  the  number  of  Brilliant  Pebbles  deployed 
varies  from  a  few  hundred  to  over  a  thousand.  These  simulations  illustrated 
that,  for  most  attack  scenarios,  the  effectiveness  of  a  proposed 
constellation  of  Brilliant  Pebbles  improved  as  the  number  of  Brilliant 
Pebbles  increased.  Simulations  also  indicate  that  a  Brilliant  Pebbles 
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constellation  orbiting  close  to  the  equator  would  be  more  effective  against 
missiles  launched  from  po'nts  in  the  Mid-East  or  Europe  and  a 
constellation  orbiting  over  the  North  and  South  Poles  would  be  more 
effective  against  attacks  from  Russia.  (See  fig.  2.1.) 

Simulations  Contain  Many 
Assumptions 

sdio  makes  assumptions  about  many  key  operational  characteristics  of  a 
Brilliant  Pebbles  constellation,  including  its  ability  to  provide  continuous 
global  surveillance,  the  length  of  time  it  takes  for  ground  control  to  give  a 
Brilliant  Pebbles  constellation  authority  to  intercept  hostile  missiles,  the 
way  in  which  Brilliant  Pebbles  in  the  constellation  are  assigned  to  targets, 
and  the  time  delay  inherent  in  the  communication  system.  The  accuracy  of 
these  assumptions  will  not  be  known  until  testing  is  completed. 

Brilliant  Pebbles’  performance  characteristics  are  also  unproven 
assumptions  because  an  integrated  and  operational  interceptor  has  not  yet 
been  bruit  and  tested.  For  the  purpose  of  simulations  that  predict  overall 
constellation  effectiveness,  SDIO  and  its  contractors  assume  that  each 
interceptor  will  perform  as  specified,  sdio  assumptions  concerning  the 
performance  characteristics  of  each  interceptor  include  its  ability  to  detect, 
track,  hit,  and  destroy  a  hostile  missile  and  its  warheads. 

Recommendations 

GAO  is  not  making  any  recommendations  in  this  report. 

Agency  Comments 

DOD  generally  agreed  with  the  information  in  GAO’s  draft  report.  The 
comments  that  it  provided  for  emphasis  and  clarification  are  included  in 
appendix  I.  It  stressed  the  importance  of  using  simulations  during  the 
demonstration  and  validation  phase  and  said  that  the  maturity  of  the 
simulations  will  be  enhanced  as  the  program  proceeds  and  that 
assumptions  will  be  modified  as  more  data  becomes  available. 
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Introduction 


In  March  1983,  President  Reagan  announced  his  decision  to  establish  an 
intensive  research  program  aimed  at  eventually  eliminating  the  threat 
posed  by  nuclear  armed  ballistic  missiles.  The  resulting  Strategic  Defense 
Initiative  (SDI)  Program  was  chartered  in  April  1984.  The  focus  of  SDI  was 
on  the  deterrence  of  Soviet  nuclear  aggression  and  reducing  our  reliance 
on  massive  retaliation  with  nuclear  weapons  for  deterrence. 

In  January  1991,  recognizing  changes  in  the  world  condition,  President 
Bush  directed  that  the  sdi  program  be  refocused  toward  providing 
protection  against  limited  ballistic  missile  strikes,  whether  deliberate, 
accidental,  or  unauthorized.  DOD’s  proposed  system  for  implementing  the 
President’s  direction  is  called  Global  Protection  Against  Limited  Strikes,  or 
GPALS.  A  GPALS  defensive  system  would  consist  of  both  surface-  and 
space-based  sensors  and  interceptors. 

The  Strategic  Defense  Initiative  Organization  (sdio)  and  its  contractors  are 
still  adapting  the  program  to  its  new  focus.  While  proposals  for  a  global 
ballistic  missile  defense  system  are  evolving  within  sdio,  Congress  has 
approved  an  early  deployment  option  to  provide  protection  for  the  United 
States  that  does  not  include  spaced-based  interceptors. 


Role  of  Brilliant 
Pebbles  in  Global 
Protection  Against 
Limited  Strikes 


Brilliant  Pebbles  is  a  proposed  concept  in  which  hundreds  of  individual 
Brilliant  Pebbles  interceptors  would  orbit  the  earth  at  relatively  evenly 
spaced  intervals  in  a  system  linked  by  communications.  Each  interceptor 
could  be  given  a  high  degree  of  autonomy  to  detect  and  attack  missiles  that 
enter  its  battle  space. 

Brilliant  Pebbles  would  be  deployed  in  space,  orbiting  at  about  400 
kilometers  (250  miles)  above  the  earth.  As  shown  in  figure  1.1,  the  Brilliant 
Pebbles  constellation  would  be  made  up  of  several  orbital  rings,  with 
several  Brilliant  Pebbles  in  each  ring,  all  traveling  at  about  5 
miles-per-second  (a  high-velocity  rifle  bullet  travels  about  a 
half-mile-per-second) . 
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The  constellation  would  be  deployed  in  such  a  way  as  to  provide 
continuous  global  coverage  for  detection  and  interception  of  ballistic 
missiles.  Once  enabled  by  human  command,  the  Brilliant  Pebbles  could 
select  their  targets  and  divert  from  their  orbits  into  the  path  of  enemy 
missiles.  The  Brilliant  Pebbles  interceptors  would  carry  no  explosives,  but 
the  force  of  their  high-speed  collision  is  expected  to  destroy  targets. 

The  flight  of  a  ballistic  missile  consists  of  four  stages:  boost,  post-boost, 
midcourse,  and  terminal.  The  boost  and  post-boost  stages  refer  to  the  first 
few  seconds  of  a  missile’s  flight  after  launch  through  the  time  its  warheads, 
or  reentry  vehicles,  and  any  decoys  are  deployed.  Midcourse  is  the 
relatively  long  period  of  time  the  reentry  vehicles  and  decoys  coast  along 
their  ballistic  trajectories  in  space  The  terminal  stage  is  the  final  minute  or 
so  when  the  reentry  vehicles  reenter  the  atmosphere  near  their  targets. 

sdio  is  determining  how  best  to  implement  the  President’s  January  1991 
direction  to  orient  proposed  missile  defenses  to  provide  protection  against 
limited  strikes.  This  involves  defining  potential  ballistic  missile  attacks,  or 
threats,  as  well  as  the  mix  of  ground-  and  space-based  interceptors  and 
sensors,  known  as  the  architecture.  The  sdio’s  Systems  Architect  and  his 
staff  have  completed  the  first  phase  of  a  study  that  lays  out  architectural 
options  to  accommodate  various  funding  levels  and  policy  decisions.  The 
SDI  program  is  currently  being  conducted  in  compliance  with  the  1972  ABM 
Treaty  and  1974  Protocol,  but  sdio  officials  have  stated  that  full-scale 
development  of  Brilliant  Pebbles  would  require  amendment  or  abrogation 
of  the  treaty.  Congress  has  urged  the  President  in  the  fiscal  year  1992 
Defense  Authorization  Act  to  discuss  with  the  Soviets  the  feasibility  and 
mutual  interests  of  amending  the  ABM  Treaty.  SDIO  says  the  administration 
has  begun  this  process  with  the  Russian  leadership. 

In  May  1991,  the  Brilliant  Pebbles  program  entered  a  50-month  phase  to 
demonstrate  and  validate  Brilliant  Pebbles’  performance  through  tests  of 
prototype  hardware,  sdio  has  since  extended  this  phase  by  18  months. 
Current  contractor  concepts  for  using  Brilliant  Pebbles  to  coimter  GPALS 
threat  scenarios  would  require  deployment  of  between  700  and  over  a 
1,000  interceptors. 
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Role  of  Computer 
Simulations  in 
Estimating  Potential 
Effectiveness 

SDlO's  estimates  of  overall  effectiveness  of  a  Brilliant  Pebbles  space-based 
interceptor  are  based  on  analytical  models  and  computer  simulations.1  At 
this  stage  of  the  program’s  development,  computer  simulations  are  the 
only  means  available  to  estimate  performance  of  a  constellation  of  Brilliant 
Pebbles  against  a  ballistic  missile  attack.  Simulations  are  used  extensively 
by  SDIO  and  its  contractors  for  basic  design  tradeoff  analyses  in  designing 
hardware.  Because  of  the  program’s  refocus  and  the  changing  ballistic 
missile  threat  estimates,  SDIO  has  not  compiled  the  results  of  all  its 
simulations  into  one  report  or  document. 

Advantages  of  Computer 
Simulations 

Computer  simulations  for  Brilliant  Pebbles  are  an  important,  though 
limited,  tool.  The  ability  to  investigate  questions  that  could  not  otherwise 
be  addressed  is  the  main  advantage  of  simulations.  They  are  extremely 
useful  in  evaluating  systems  that  do  not  yet  exist  or  exist  only  in  limited 
numbers.  However,  simulations  are  of  necessity  based  on  assumptions  and 
estimates  about  the  design,  performance,  and  availability  of  technologies 
for  Brilliant  Pebbles.  These  assumptions  are  by  nature  simplified  and 
cannot  be  as  complex  as  reality. 

sdio  considers  credible  simulations  to  be  an  important  part  of  early 
effectiveness  assessments  that  support  m^jor  milestone  decision  points  in 
the  acquisition  process.  Moreover,  sdio’s  reliance  on  simulations  increases 
as  weapon  systems  become  more  complex  and  expensive  to  test.  This  is 
especially  true  of  ballistic  missile  defenses,  where  constraints  such  as  cost, 
safety,  the  ability  to  accurately  portray  threats,  and  treaty  limitations 
combine  to  preclude  thorough  evaluation  through  field  testing  alone. 

Assumptions  Limit 
Usefulness  of  Computer 
Simulations 

Although  simulations  are  useful  tools,  they  rely  on  data  that  may  be 
incomplete  and  assumptions  that  may  be  inaccurate.  Constructing  an 
accurate  simulation  requires  that  the  behavior  of  what  is  being  simulated 
be  well  understood;  there  is  a  great  danger  in  accepting  the  results  of 
computer  simulations  as  representing  reality,  rather  than  using  them  as 
design  tools.  The  accuracy  of  a  simulation  can  only  be  checked  by 
comparisons  with  measured  results  in  the  real  world,  a  process  called 
“validation.”  The  need  for  validation  was  illustrated  recently  in  news 
accounts  of  a  m^jor  air  crash. 

'in  this  report  we  will  use  the  term  simulation  to  refer  to  both  analytical  models  and  computer 
simulations. 
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An  Investigation  of  the  crash  of  a  Boeing  767  in  Thailand  in  May  1991 
revealed  that  even  a  highly  sophisticated  flight  simulator  of  an  operational 
system  was  limited  by  assumptions.  The  767  flight  simulator  initially  used 
to  recreate  the  crash  indicated  that  a  pilot  could  maintain  control  of  the 
airplane  if  a  thrust  reverser  accidentally  deployed  in  flight.  However,  as 
confirmed  by  a  Federal  Aviation  Administration  official,  later  data  from 
wind  tunnel  tests  indicated  that  the  simulation  software  was  faulty  and  had 
to  be  revised  to  accurately  represent  the  aircraft’s  behavior.  After 
modification,  the  simulator  showed  the  aircraft  rolling  onto  its  back  and 
going  into  a  dive  before  corrective  actions  were  possible. 

SDio's  simulations  of  the  effectiveness  of  a  Brilliant  Pebbles  constellation 
are  not  as  mature  in  their  development  as  a  flight  simulator.  Early 
simulation  results  give  some  confidence  that  the  proposed  design  of  the 
Brilliant  Pebbles  and  its  constellation  can  function  without  violating 
fundamental  understandings  of  physics,  geometry,  and  technology.  SDio’s 
simulations  currently  assume  an  interceptor  can  be  developed  to  meet 
performance  specifications  and  that  developers  can  overcome  the  many 
technology  and  engineering  challenges  contained  in  Brilliant  Pebbles’ 
numerous  innovative  features. 

Both  the  simulations  being  used  and  the  assumptions  made  about  technical 
performance  contribute  to  the  limitations  of  Brilliant  Pebbles’  effectiveness 
estimates  (see  chap.  3).  DOI)  has  recognized  these  limitations.  In  1987  SDIO 
commissioned  an  internal  review  team  to  visit  contractors  using 
simulations  in  the  design  and  evaluation  of  strategic  defense  systems.  The 
responsible  sdio  otficial  said  he  proposed  the  study  after  realizing  that  the 
effectiveness  results  in  contractor  briefings  meant  little  unless  the 
supporting  simulations  were  fully  understood. 

The  review  team  examined  10  simulations,  some  of  which  have  been  used 
to  evaluate  Brilliant  Pebbles,  and  concluded  that  none  was  completely 
acceptable  as  a  general-purpose  evaluation  tool.  They  further  concluded 
that  predictions  based  on  simulations  were  not  sufficiently  credible  to 
serve  as  the  sole  basis  for  SDi  program  decisions. 


Many  Models  Of  Varied  Detail  SDIO  and  its  supporting  contractors  use  many  simulations  with  varying 
Currently  Being  Used  levels  of  detail  to  predict  effectiveness.  For  example,  SDIO  and  its 

contractors  use  battle  engagement  simulations,  also  called 
“force-on-force”  simulations.  These  represent  the  performance  of  the 
entire  Brilliant  Pebbles  constellation  against  attacking  ballistic  missiles  and 
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tabulate  results  in  terms  of  the  number  of  reentry  vehicles  intercepted. 
Battle  engagement  simulations  are  used  primarily  to  vary  the  number  of 
Brilliant  Pebbles  in  a  constellation,  or  to  test  the  effects  of  different  battle 
management  schemes. 

Simulations  of  a  single  interceptor  component  or  function  are  at  a  greater 
level  of  detail.  For  example,  simulations  may  represent  how  a  interceptor 
would  use  its  sensors  to  scan  the  earth  looking  for  ballistic  missile  launches 
or  how  an  interceptor  would  guide  itself  to  a  target.  SDIO  uses  these  more 
detailed  simulations  to  build  confidence  in  the  results  of  the  overall  battle 
engagement  simulations. 


Validated  and  Verified 
Simulations  Are  to  Be 
Developed 


The  accuracy  of  Brilliant  Pebbles’  simulations  has  not  been  validated  by 
test  results  or  verified  by  formal  reviews  of  the  simulations  themselves. 
However,  test  results  will  become  increasingly  available  as  the  program 
progresses  through  the  demonstration  and  validation  phase.  Currently, 

SDIO  compares  various  simulation  results  to  assure  itself  that  Brilliant 
Pebbles’  simulations  are  reasonably  accurate. 

SDIO  refers  to  formal  simulation  reviews  as  confidence  assessments. 
Conducting  a  confidence  assessment  is  a  systematic  way  of  examining  the 
credibility  of  a  computer  simulation.  In  confidence  assessments, 
independent  reviewers  examine  a  simulations’s  conceptual  base,  determine 
the  validity  of  the  input  data,  and  identify  the  simulation’s  limitations. 

SDIO  funded  confidence  assessment  work  through  fiscal  year  1990. 
According  to  SDIO  and  it  contractors,  formal  confidence  assessments  were 
suspended  in  fiscal  year  1991  due  to  funding  constraints  and  because  the 
architecture  for  a  ballistic  missile  defense  system  and  the  related  design 
features  of  the  interceptor  were  too  dynamic  to  permit  detailed  verification 
of  current  Brilliant  Pebbles’  simulation  models. 

In  lieu  of  formal  confidence  assessments,  SDIO  and  its  contractors  compare 
the  results  from  the  simulations  done  by  one  group  with  results  from 
others.  Beginning  with  the  Space  Based  Architecture  Study  in  1989,  SDIO 
and  contractors  have  used  this  method  to  reassure  themselves  that  Brilliant 
Pebbles’  simulations  are  as  accurate  as  possible  at  this  stage  of  the 
program’s  development.  As  discussed  in  chapter  2,  sdio  plans  to 
standardize  the  simulation  process. 
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Introduction 


Objectives,  Scope,  and 
Methodology 


The  Chairman,  Senate  Armed  Services  Committee,  requested  that  we 
identify  the  critical  assumptions  of,  and  uncertainties  in,  estimates  of  the 
effectiveness  of  Brilliant  Pebbles.  As  agreed  to  with  his  office,  we  limited 
our  work  to  the  effectiveness  estimates  for  a  gpals  mission.  Our  objectives 
were  to  examine  SDiO's  calculation  of  Brilliant  Pebbles’  contribution  to  the 
GPALS  mission  and  identify  critical  assumptions  and  uncertainties  contained 
in  SDiO's  analyses  of  Brilliant  Pebbles’  effectiveness. 

We  met  with  officials  from  sdio  and  contractors  working  on  the  program. 
We  examined  studies  of  the  Brilliant  Pebbles  to  develop  information  about 
critical  issues  that  sdio  must  address.  We  also  examined  studies  that 
discussed  implications  of  the  move  to  a  gpals  system.  We  did  not  evaluate 
technical  aspects  of  the  Brilliant  Pebbles  to  determine  if  it  would  work  as 
specified. 

We  conducted  our  work  from  January  through  November  1991  in 
accordance  with  generally  accepted  government  auditing  standards.  DOD 
provided  written  comments  on  a  draft  of  this  report.  Its  comments  are 
included  in  appendix  I. 
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Based  on  its  computer  simulations,  SDIO  believes  that  Brilliant  Pebbles’ 
interceptors  would  contribute  significantly  to  the  gpals  mission  of 
protecting  the  United  States  and  our  allies  from  limited,  unauthorized,  or 
accidental  attacks,  sdio  estimates  that  a  space-based  constellation  of 
between  600  and  1,000  interceptors  could  have  significant  intercept 
capability  against  a  broad  class  of  missiles  with  ranges  longer  than  600 
kilometers. 


Effectiveness  Depends 
on  Nature  of  Ballistic 
Missile  Attacks 


To  assess  effectiveness,  dod  must  decide  on  the  types  and  numbers  of 
ballistic  missiles  the  Brilliant  Pebbles  constellation  must  intercept,  dod’s 
decision  will  be  based  in  large  part  on  the  input  from  SDiO's  Threat  Working 
Group,  which  has  identified  over  40  hypothetical  attack  scenarios,  or 
threats,  including  short-,  intermediate-,  and  long-range  ballistic  missile 
attacks  over  the  entire  globe  and  submarine  launched  attacks  against  the 
United  States.  Brilliant  Pebbles’  effectiveness  is  contingent  on  where  the 
ballistic  missiles  originate,  where  they  are  headed,  and  their  positions 
relative  to  the  orbital  positions  of  the  Brilliant  Pebbles. 


The  design  of  the  interceptors  would  prevent  them  from  entering  the 
earth’s  atmosphere,  and  as  such,  they  would  have  difficulty  in  countering 
very  short-range  ballistic  missile  attacks  because  these  missiles’ 
trajectories  do  not  go  abo/e  the  atmosphere. 


As  currently  designed,  Brilliant  Pebbles  could  not  intercept  missiles  with 
ranges  of  less  than  400  to  600  kilometers  or  those  with  altitudes  less  than 
80  to  100  kilometers.  This  means  that  some  missiles  currently  owned  by 
Third  World  countries  could  not  be  attacked  by  Brilliant  Pebbles.  However, 
only  a  few  of  the  more  than  40  threat  scenarios  identified  employ  such 
short-range  missiles.  Intercepting  cruise  missiles  and  aircraft  is  not  part  of 
Brilliant  Pebbles’  mission. 


Deployment  Decisions 
Impact  Brilliant 
Pebbles  Effectiveness 


The  number  of  Brilliant  Pebbles  deployed  and  the  way  in  which  they  orbit 
the  earth  has  a  direct  effect  on  the  performance  of  the  Brilliant  Pebbles 
constellation  against  a  given  threat. 

Since  the  refocus  on  GPALS,  SDIO  has  performed  simulations  in  which  the 
number  of  Brilliant  Pebbles  in  orbit  varies  from  a  few  hundred  to  over  a 
thousand.  These  simulations  illustrated  that  for  most  attack  scenarios,  the 
effectiveness  of  a  hypothetical  Brilliant  Pebbles  constellation  improved  as 
the  number  of  Brilliant  Pebbles  in  orbit  was  increased. 
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The  orbital  inclination  of  the  Brilliant  Pebbles  also  has  a  significant  effect 
on  performance.  Orbital  inclination  is  the  angle  that  the  Brilliant  Pebbles 
orbit  at  in  relation  to  the  equator.  A  ring  of  Brilliant  Pebbles  in  orbit  at  the 
equator  would  have  an  inclination  of  zero  degrees  and  a  ring  in  a  polar 
orbit  would  have  an  inclination  of  90  degrees. 

SDlo’s  simulations  indicate  that  there  are  different  orbital  inclinations  that 
optimize  the  effectiveness  of  a  Brilliant  Pebbles  constellation  against  each 
of  the  individual  threats.  For  example,  a  Brilliant  Pebbles  constellation  in  a 
lower  orbital  inclination  would  be  more  effective  against  missiles  launched 
from  points  in  the  Mid-East  or  Europe  and  a  constellation  in  a  higher 
orbital  inclination  would  be  more  effective  against  attacks  from  Russia. 
However,  after  the  Brilliant  Pebbles  are  deployed,  their  orbital  inclinations 
cannot  be  changed.  To  provide  continuous  global  coverage  for  both  the 
detection  and  interception  of  enemy  ballistic  missiles,  at  least  a  portion  of 
the  constellation  will  most  likely  be  deployed  at  near-polar  orbits. 
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Figure  2.1 :  Different  Coverage  Offered  by  Different  Inclinations 
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Estimated 

Effectiveness  Based  on 
Computer  Simulations 


SDlO's  estimates  of  effectiveness  were  based  on  computer  simulations  of 
Brilliant  Pebbles’  performance  using  a  certain  number  of  interceptors 
deployed  in  a  particular  constellation  against  hypothetical  threats,  such  as 
short-,  intermediate-,  and  long-range,  and  submarine-launched  attacks.  As 
of  December  1991,  sdio  had  not  evaluated  performance  against  all  the 
threat  scenarios,  sdio  told  us  subsequently  that  they  completed  this 
evaluation. 


One  type  of  threat  scenario  that  SDIO  did  examine  was  two  short-range  A1 
Abbas  missile  attacks  by  Iraq.  One  attack  was  against  the  Arabian 
peninsula,  which  showed  that  Brilliant  Pebbles  would  be  able  to  intercept 
most  of  the  missiles.  In  the  other  attack,  sdio  ran  a  simulation  showing  how 
Brilliant  Pebbles  would  have  performed  against  the  Iraqi  missiles  launched 
during  Operation  Desert  Storm.  The  simulation  used  Air  Force  data  for  79 
of  the  actual  launches,  the  launch  points,  aim  points,  and  maximum  heights 
reached  by  the  missiles,  and  showed  that  the  Brilliant  Pebbles  constellation 
in  a  high  orbital  inclination  would  have  intercepted  about  69  of  the 
missiles. 


To  estimate  Brilliant  Pebbles’  effectiveness  against  intermediate-range 
missiles,  SDIO  ran  a  simulation  of  Brilliant  Pebbles’  performance  against  an 
intermediate-range  attack  on  Great  Britain  by  Libya.  It  showed  that  all  of 
the  missiles  would  be  intercepted  by  the  Brilliant  Pebbles. 

Another  simulation  was  run  to  determine  Brilliant  Pebbles’  effectiveness 
against  a  limited  Soviet  long-range  attack  using  land-based  intercontinental 
ballistic  missiles.  The  simulation  showed  that  Brilliant  Pebbles  would  be 
able  to  intercept  most  of  the  missiles. 

SDIO  also  ran  a  simulation  to  determine  Brilliant  Pebbles’  effectiveness 
against  an  attack  launched  by  a  submarine  off  the  U.S.  coast.  It  showed 
that  Brilliant  Pebbles  would  be  able  to  intercept  almost  two-thirds  of  the 
missiles  launched. 


SDIO  Plans  to  Develop 
Three  New  Simulations 
for  Brilliant  Pebbles 


Currently,  sdio  and  its  contractors  use  many  different  simulations  to 
estimate  Brilliant  Pebbles'  effectiveness.  SDIO  plans  to  standardize  the 
simulation  process  by  requiring  each  of  the  two  contractors  for  the 
Brilliant  Pebbles  program  to  design  and  install  a  Brilliant  Pebbles 
simulation  at  the  National  Test  Bed’s  hub,  the  National  Test  Facility  in 
Colorado  Springs,  Colorado,  by  1993.  In  addition  to  these  two  simulations, 
SDIO  and  the  National  Test  Bed  staff  will  develop  and  install  a  third  Brilliant 
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Pebbles  simulation.  This  simulation  is  expected  to  be  comprehensive  and 
highly  detailed  and  will  serve  as  a  basis  for  comparison  for  the  two 
contractors’  simulations. 

According  to  SDio,  the  two  contractors’  simulations  of  Brilliant  Pebbles  will 
utilize  super-computers,  which  can  process  larger  amounts  of  information 
and  thus  do  more  detailed  simulations.  Most  of  the  Brilliant  Pebbles 
simulations  done  to  date  have  been  run  on  relatively  small  computers  and 
some  have  been  modifications  of  simulations  used  for  other  weapons 
programs  such  as  air-to-air  missile  systems  and  the  prior  Space-Based 
Interceptor  program. 

SDio  established  an  Algorithm  Working  Group  as  a  way  to  oversee 
simulation  efforts.  It  is  made  up  of  experts  and  technical  specialists  who 
come  together  to  analyze  problems  and  to  review  simulations  and  their 
underlying  performance  assumptions.  As  part  of  its  draft  charter,  the 
Algorithm  Working  Group  is  to  identify  problem  areas  where  intensive 
simulation  development  efforts  are  required.  The  group  is  also  concerned 
with  ensuring  that  simulations  are  validated  with  test  results. 
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The  simulations  SDIO  uses  to  estimate  the  effectiveness  of  Brilliant  Pebbles 
are  relatively  immature  and  must  use  many  unproven  assumptions  about 
the  performance  and  operation  of  the  constellation.  For  example,  in  its 
simulations  sdio  makes  assumptions  about  probabilities  that  an  interceptor 
will  detect,  track,  collide,  and  destroy  the  missile  or  reentry  vehicle.  SDIO 
also  assumes  that  the  Brilliant  Pebbles  constellation  will  receive  timely 
weapons  release  authority  from  a  ground  station  and  that  the  battle 
management  scheme  will  ensure  that  the  right  interceptors  will  attack  the 
right  targets. 

As  Brilliant  Pebbles  development  progresses,  sdio  plans  to  replace  early 
assumptions  about  Brilliant  Pebbles’  performance  with  data  obtained 
through  testing  and  continued  simulation.  Ground  and  space-flight  tests 
will  be  used  to  validate  performance  assumptions  for  a  single,  or  just  a  few, 
interceptors.  Computer  simulations  will  continue  to  be  used  to  predict  both 
the  performance  of  a  single  interceptor  and  of  an  entire  constellation  of 
interceptors  against  various  ballistic  missile  attacks. 


Effectiveness 
Estimates  Are  Based 
on  Many  Assumptions 


In  addition  to  assumptions  about  threat  and  the  configuration  of  the 
Brilliant  Pebbles  constellation  discussed  in  chapter  2,  sdio  makes 
assumptions  about  other  operational  characteristics  of  the  constellation 
and  the  performance  characteristics  of  each  Brilliant  Pebbles  interceptor. 

Simulations  cannot  prove  that  the  Brilliant  Pebbles  system  will  work.  Both 
SDIO  officials  and  contractors  stated  that  only  testing  can  do  that.  The 
Director  of  Integration  in  the  Brilliant  Pebbles  program  office  said  that 
most  simulations  represent  only  the  most  basic  characteristics  of  a  Brilliant 
Pebble.  The  Director  of  sdio’s  Architecture  Integration  Study  stated  that 
the  simulations  used  by  his  group  assume  that  Brilliant  Pebbles  will 
acquire,  track,  and  intercept  targets  perfectly  and  the  results  represent  the 
best  that  could  possibly  be  expected  from  a  Brilliant  Pebbles  constellation. 
According  to  sdio,  a  wide  range  of  assumption  values  are  being  used 
during  the  early  design  phase  of  Brilliant  Pebbles  to  examine  potential 
system  performance  and  sensitivity. 


Assumptions  Concerning  sdio  makes  assumptions  about  the  operational  characteristics  of  the 
Operational  Characteristics  Brilliant  Pebbles  constellation,  including  continuous  global  surveillance, 

the  length  of  time  it  takes  for  ground  control  to  give  the  constellation 
authority  to  intercept  hostile  missiles,  the  way  in  which  Brilliant  Pebbles  in 
the  constellation  are  assigned  to  targets,  and  the  time  delay  inherent  in  the 
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communication  system.  The  validity  of  these  assumptions  has  not  been 
demonstrated  by  testing. 


One  of  the  requirements  for  the  Brilliant  Pebbles  constellation  is  that  it 
provide  continuous  global  surveillance  and  detection  of  ballistic  missile 
launches.  To  determine  the  best  way  to  meet  this  requirement,  SDio  runs 
simulations  that  calculate  surveillance  coverage  based  on  assumptions 
about  sensor  performance  characteristics  including  acquisition  range,  field 
of  view,  resolution  of  multiple  targets,  and  line-of-sight  constraints  such  as 
blinding  by  the  sun.  Simulations  can  vary  the  range  at  which  the  sensors 
detect  targets  and  estimate  the  effects  of  the  sensors  losing  targets  that  are 
close  together  or  whose  tracks  cross.  For  line-of-sight  constraints,  the 
cloud  cover  is  set  at  different  altitudes  and  the  sun’s  blinding  effect  on 
certain  sensors  can  also  be  simulated. 


The  time  it  takes  for  the  Brilliant  Pebbles  constellation  to  get  weapons 
release  authority  from  ground  control  is  a  critical  factor.  SDIO’s  simulations 
show  that  effectiveness  of  the  constellation  decreases  as  the  time  required 
to  receive  weapons  release  authority  increases.  SDIO  says  these  results  will 
have  a  significant  impact  on  the  design  of  the  battle  management  system. 
Brilliant  Pebbles  related  command  and  control  simulations,  often  referred 
to  as  war  games,  have  been  conducted  at  the  National  Test  Facility. 


Assumptions  concerning  battle  management  have  an  impact  on 
effectiveness  calculations.  The  objective  of  battle  management  is  to  assign 
interceptors  to  the  right  targets  based  on  various  strategies,  such  as 
maximizing  the  number  of  reentry  vehicles  killed  or  minimizing  over-  or 
under-assignment  of  Brilliant  Pebbles  to  targets.  Battle  management 
decisions  can  be  made  by  ground  control  or  autonomously  by  each 
interceptor  based  on  what  it  can  see  and  what  it  knows  about  the  positions 
of  surrounding  Brilliant  Pebbles. 

To  perform  the  battle  management  function  autonomously,  the  interceptor 
needs  to  determine  where  the  target  came  from,  what  type  of  target  it  is, 
and  where  it  is  headed.  The  interceptor  must  also  be  able  to  assign  a 
unique  label  to  each  missile  it  sees  in  order  to  keep  track  of  many  missiles 
at  once.  This  problem  is  referred  to  as  correlation,  and  SDio  contractors 
continue  to  work  on  it.  sdio  has  identified  tracking  and  target  identification 
as  a  ircuor  technical  challenge. 
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Communications  In  many  simulations,  communications  are  assumed  to  be  perfect  and 

timely.  Communications  from  the  ground  to  the  constellation  are 
performed  via  a  trellis  network:  a  message  is  passed  from  the  ground  to 
whichever  interceptor  is  in  position  to  receive  it.  That  interceptor  passes 
the  message  on  to  its  neighboring  interceptors,  each  of  which  repeats  the 
process  until  the  entire  constellation  receives  the  message,  sdio  estimates 
that  this  process  will  take  only  a  few  seconds. 

SDIO  has  stated  that  communications  from  ground  control  to  the  Brilliant 
Pebbles  constellation  will  most  likely  utilize  high-frequency  radio  signals. 
However,  both  radio  frequency  and  laser  communication  systems  are  being 
considered  for  the  space-to-space  communication  link  from  one 
interceptor  to  another,  sdio  considers  both  the  high-frequency  radio  and 
laser  communication  systems  to  be  major  technical  challenges. 


Assumptions  Concerning 
Each  Interceptor’s 
Performance 


Interceptor  performance  characteristics  are  unproven  assumptions  at  this 
point  because  an  integrated  and  operational  interceptor  has  not  yet  been 
built  and  tested.  For  the  purpose  of  overall  system  effectiveness 
simulations,  SDIO  and  contractors  assume  that  a  single  interceptor  will 
perform  as  specified.  SDIO  assumptions  concerning  the  performance 
characteristics  of  each  interceptor  include  its  ability  to  detect,  track,  hit, 
and  destroy  the  target. 


Detection  and  Tracking  of  To  predict  where  its  target  is  going,  an  interceptor  must  first  detect  and 

Targets  track  it  for  a  number  of  seconds.  However,  the  longer  the  interceptor 

tracks  its  target,  the  less  time  it  will  have  to  perform  the  intercept. 
Simulations  have  shown  that  Brilliant  Pebbles’  ability  to  track  a  target  is 
dependent  not  only  on  track  time,  but  also  on  its  position  in  relation  to  the 
target,  the  speed  at  which  the  sensor  can  process  information,  and  sensor 
measurement  errors.  If  the  interceptor  has  a  large  tracking  error,  it  will 
have  greater  difficulty  intercepting  the  target. 


Interceptor  Fly-Out  to  Target  Target  engagement  simulations  are  often  broken  down  into  several  phases, 

including  most  of  the  interceptor  fly-out  phase  referred  to  as  the 
mainchase  and  the  final  seconds  of  the  fly-out  phase  referred  to  as  the 
end-game.  Contractors  use  the  most  detailed  simulations  available  to 
analyze  these  phases. 

These  simulations  are  based  on  assumptions  about  such  propulsion 
characteristics  as  how  fast  the  interceptor  will  fly,  how  far  it  can  fly,  and 
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how  much  it  can  maneuver.  Both  demonstration  and  validation  contractors 
have  established  performance  goals  for  their  baseline  Brilliant  Pebbles 
designs.2  According  to  sdio,  developing  an  interceptor  to  meet  these  goals 
will  be  technically  challenging. 


Many  of  the  uncertainties  associated  with  the  performance  of  an 
interceptor  are  contained  in  what  is  referred  to  as  the  single-shot 
probability  of  kill  factor  (SSPK).  SSPK  represents  the  probability  that  a 
single  interceptor  has  of  successfully  hitting  and  destroying  its  assigned 
target.  For  example,  an  interceptor  with  a  SSPK  of  .6  would  hit  and  destroy 
its  target,  on  average,  60  percent  of  the  time  while  an  interceptor  with  a 
sspk  of  .9  would  hit  and  destroy  its  target  90  percent  of  the  time. 

In  many  of  the  simulations  done  to  date,  SDIO  and  its  contractors  have 
typically  used  a  SSPK  of  .8  or  .9.  In  some  cases,  contractors  have  used  a 
SSPK  of  1  in  order  to  define  the  best  the  constellation  could  do  against  a 
given  attack. 

Contractors  told  us  that  the  overall  estimated  SSPK  of  an  interceptor  '•ould 
be  lower  than  .8  or  .9.  A  lower  SSPK  will  result  in  fewer  targets  destroye  d. 
SSPK  is  the  product  of  a  number  of  factors  such  as  the  probabilities  cf  the 
interceptor  successfully  detecting,  tracking,  hitting,  and  destroying  a 
target.  For  instance,  if  there  were  five  factors  that  made  up  SSPK  and  each 
had  a  value  of  .9,  the  resulting  SSPK  value  would  be  .59,  as  shown  below. 


SSPK  =  .9  x  .9  x  .9  X  .9  x  .9 
SSPK  =  .59 

Perfect  performance  is  being  assumed  for  some  of  these  factors.  For 
example,  the  probability  that  the  interceptor  will  successfully  separate  and 
fly  away  from  its  lifejacket3  has  been  assumed  to  be  1 .  In  another  instance  a 
contractor  told  us  that  there  were  no  hardware  reliability  figures  factored 
into  the  SSPK  he  used,  which  meant  that  the  actual  sspk  could  be  lower. 


In  its  simulations,  sdio  assumes  that  a  hit  renders  the  missile  harmless. 
sdio  believes  that  Brilliant  Pebbles  can  hit  hostile  ballistic  missiles  and 


^ese  data  at  e  not  supplied  because  they  are  classified. 

3The  lifejacket  houses  a  single  interceptor  and  its  supporting  systems. 
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reentry  vehicles,  but  it  does  not  know  if  Brilliant  Pebbles  will  in  all  cases 
render  the  target  harmless.  This  problem  is  referred  to  as  lethality,  and  it 
involves  where  the  intercept  occurs  (preferably  over  the  attacker  rather 
than  over  the  intended  target),  and  how  much  of  the  missile’s  destructive 
capability  is  destroyed. 

The  lethality  problem  was  highlighted  in  the  Gulf  War  when  Patriot 
ground-based  interceptors  broke  incoming  warheads  into  pieces  above 
their  intended  targets,  but,  in  some  cases,  the  falling  pieces  still  did 
considerable  damage,  dod  is  concerned  with  assuring  that  nuclear, 
chemical,  biological,  or  conventional  warheads  would  be  destroyed  in  such 
a  manner  as  to  avoid  significant  damage  from  the  resulting  debris,  if  any. 

SDIO  has  stated  that  it  needs  to  dc  more  testing  to  fully  understand  Brilliant 
Pebbles’  lethality.  The  demonstration  and  validation  program  is  planned  to 
provide  a  wealth  of  information  on  Brilliant  Pebbles’  actual  lethality. 
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Comments  From  the  Department  of  Defense 


DEPARTMENT  OF  DEFENSE 
STRATEGIC  DEFENSE  INITIATIVE  ORGANIZATION 
WASHINGTON.  DC  20301-7100 


February  26,  1992 


Mr.  Franlc  C.  Con  man 
Assistant  Comptroller  Ganeral 

National  Security  and  International  Affairs  Division 
U.S.  General  Accounting  Office 
Washington,  D.C.  20S48 

Dear  Mr.  Conahan: 

This  is  the  Department  of  Defense  response  to  the  General 
Accounting  Office  (GAO)  Draft  Report  entitled — ‘STRATEGIC  DEFENSE 
INITIATIVE:  Estimates  of  Brilliant  Pebbles  Effectiveness  Are 

Based  on  Many  Unproven  Assumptions,*  dated  February  7,  1992  (GAO 
Code  392600/OSD  Code  8939).  Although  the  Department  generally 
concurs  with  the  report,  the  following  comments  are  offered  for 
emphasis  and  clarification. 

The  use  of  simulation  technology  in  the  development  of  ths 
Brilliant  Pebbles  program  is  most  important  in  evaluating  and 
developing  the  Brilliant  Pebbles  hardware.  An  integrated  testing 
program  is  established  to  address  the  critical  technical  and 
operational  issues,  and  the  continuing  testing  efforts  will 
provide  validation  of  simulations  and  models.  Simulations  can 
reduce  test  costs  when  used  in  conjunction  with  testing  in  a 
synergistic  combination. 

Brilliant  Pebbles  simulation  activities  are  consistent  with 
a  program  in  the  demonstration  and  validation  phase.  The 
‘maturity’  of  Brilliant  Pebbles  simulations  will  change  and  be 
enhanced  with  Improvement  in  the  design  of  primary  system 
hardware  prototypes.  It  is  crucial  that  the  simulation  efforts 
provide  sufficiency  to  allow  the  program  to  proceed  to  the  next 
milestone. 

The  report  indicates  that  simulations  may  rely  un  data  that 
are  incomplete  and  assumptions  that  may  be  inaccurate.  That  does 
not  limit  the  simulation  usefulness.  The  Strategic  Defense 
Initiative  Organization  has  relied  on  an  arduous  engineering 
assessment  tempered  by  real-world  experience  to  arrive  at  a 
working  hypothesis.  Assumptions  are  based  upon  a  combination  of 
the  understanding  of  the  system  operation,  operating 
characteristics,  and  engineering  analysis.  An  more  data  becomes 
available,  assumptions  are  modified  as  necessary.  Additionally, 
the  Strategic  Defenso  Initiative  Organization  has  relied  upon  the 
best  available  threat  information,  as  found  in  the  most  current 
intelligence  scenarios. 

It  also  should  be  noted  that  some  of  ths  assumptions 
reflect  validated  operational  requirements.  The  acquisition 
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process  requires  an  evaluation  of  system  capability 
to  meet  those  requirements.  The  Strategic  Defense  Initiative 
Organization  does  not  randomly  choose  parameters.  Operational 
requirements  are  matched,  to  the  greatest  extent  possible,  to 
system  performance  assumptions.  Furthermore ,  it  should  be 
recognized  that  system  effectiveness  also  is  a  function  of 
selected  tactics  and  that  the  user,  U.S.  Space  Command,  is  deeply 
involved  in  the  development  of  operational  employment,  strategy, 
and  tactics. 

In  addition,  some  of  the  target  and  lethality  comparisons 
with  the  performance  of  the  PATHIOT  in  the  Gulf  War  are  not 
appropriate.  Brilliant  Pebbles  is  being  designed  to  intercept  so 
far  away  from  the  target  that  falling  debris  would  not  be  a 
problem  at  the  target  location.  The  ability  to  predict  the 
effects  of  an  interceptor  on  an  Incoming  missile  or  missile 
warhead  have  not  been  fully  defined  and  tasted  for  the  Strategic 
Defense  Initiative  Organization. 

The  Department  of  Defense  appreciates  the  opportunity  to 
comment  on  the  draft  report. 

HENRY  F.  COOPER 
Director 
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Major  Contributors  to  This  Report 


National  Security  and 
International  Affairs 
Division 

Washington,  D.C. 


Brad  Hathaway,  Associate  Director 
J.  Klein  Spencer,  Assistant  Director 
Charles  A.  Walter  III,  Assignment  Manager 


San  Francisco  Regional 
Office 


Frank  Graves,  Regional  Manager  Representative 
Thomas  M.  James,  Evaluator-in-Charge 
Kathryn  J.  Mathisen,  Evaluator 


(S92M0) 
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